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ABSTRACT 

The aim of this study was to assess gamification as a method of experiential learning theory (ELT) 
on student motivation and self-efficacy to perform System Engineering/Information Assurance (IA) 
tasks. The study was a basic qualitative method, whereby data was collected via semi-structured 
interview and then analyzed for recurring themes and patterns. The students involved in the study 
were undergraduate students enrolled in system administration and security courses. We 
introduced ELT in early stages of curriculum in place of commonly used didactic methods of 
delivering theory. We compared the themes found in increased ELT classes with past didactic 
sections of the same courses. Data analysis revealed that increasing ELT in IA coursework at all 
levels of the curriculum increased both student intrinsic motivation and self-efficacy. This paper 
outlines gamification pedagogy used in 200 and 300 level postsecondary courses of system 
administration. Gathered results indicated high intrinsic motivation and self-efficacy from the 
students 96 interviewed. The paper will also present examples of gamification ELT lessons at 
each level of undergraduate study. 
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INTRODUCTION 


oday most university students sit and listen while taking notes as a professor delivers a lecture. This 
fy/ is traditional didactic education that is instructor centered. While there could be 10 to 100 human 

beings in the room, there is little to no human interaction. Exam time is the first chance most 
professors using this pedagogy can gauge how well students digested the delivered lecture information. Here, 
student motivation is extrinsic, focused on grade, employment or task rather than intrinsic motivation, which focuses 
on satisfaction, or pleasure in performing a task (Lei, 2010). This stems from a belief system that learning should 
feel much like work (Vesterbacka, 2013). 


Research literature is heavy in claiming that American schools are not teaching enough, are not rigorous 
enough, and offer too much play (Ravitch, 2013; Sahberg, 2011; Zwaagsta, et al, 2010). On the contrary, American 
Schools are not playing enough and we are working too hard. The didactic approach that is so common in post¬ 
secondary education needs to be integrated with doing (Kolb, 1984). Experiential learning theory, or ELT, is active 
learning where the learning is student-centered activity. Gamification, an offshoot of ELT, is the combination of 
game components such as score, challenge, and achievement with learning objectives in an effort to motivate and 
engage the student (Deterding et al,2011). 

The extrinsically motivated student attends class because they have to for attendance, grade or other 
external motivator (Ryan & Deci, 2010). The intrinsically motivated (IM) student comes to class because they 
hunger for more information, they want to be there, they are involved and their self-efficacy soars. Classrooms must 
become student centric to reach IM goals, and gamification of learning objectives can be used to reach that target. 
Research on using gamification to drive educational goals outlined by Ryan & Deci is sparse in System 
Engineering/Information Assurance (IA) courses (2010). The majority of studies (Engler, 2012; Muntean, 2011; 
Renaud & Wagner, 2011) focus on the use of commercially available games, however this research investigates the 
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creation of didactic learning objectives blended with gaming theory to increase intrinsic motivation and self- 
efficacy. 


The problem of education is that we must convert student extrinsic motivation (EM) into intrinsic 
motivation (IM) in order to increase student self-efficacy (SE) to explore, participate and to reach true knowledge 
gain and nurture new innovative thought (Bandara 1977; Kolb 1984). In this research, we study how the use of 
gamification pedagogy across 200 and 300 level postsecondary courses in Information Assurance (IA) affect student 
IMandSE. 

Experiential Learning Theory, Motivation and Gamification 

Experiential learning theory (ELT) 

ELT is not a new pedagogy as it was developed by Dr. David Kolb (1984) and built on the foundation 
methods established by education theory pioneers John Dewey, Kurt Lewin, and Jean Piaget. Kolb’s ELT describes 
learning as a process whereby thoughts are formed then re-formed through experience, thus creating new knowledge 
and deeper understanding (Kolb, 1984). ELT, at the very core, is hands-on learning pedagogy that is student centric. 
Muntean (2011) further defined hands-on learning as an educational process that through practice, creates aptitude 
and deeper understanding that can lead to an increase in student intrinsic motivation. The ancient Chinese proverb, 
“Teach me and I will forget. Show me and 1 may remember. Involve me and I’ll understand...” captures the central 
theme of why experiential learning is critical to education (unknown source). 

ELT is commonplace in IA education research (Stockman & Nylan, 2112; Cooper et al, 2010), however 
using gamification as the underpinning pedagogy to raise IM and SE in IA is sparse. Gamification is commonly 
used in business programs in the form of case study and in elementary schools to engage new learners, but is just 
breaking surface in other disciplines. Gamification does not simply imply creating a game, it is a pedagogy used to 
make the student more engaged without deflation of educational credibility (Muntean, 2011). 

Motivation and self-efficacy 

Motivation to learn is seen in the form of student persistence, curiosity, and performance (Lei, 2010). 
Intrinsic motivation (IM) is defined as participating for pleasure, or satisfaction derived from performing an act. As 
IM increases, self-efficacy (SE) to complete tasks is learned and active processes begin which lead to deeper 
understanding and the creation of aptitude (Chentanez at al., 2004). Bandura defined SE as personal belief of one’s 
own capability to execute strategy to attain designated goals (1977). Zimmerman (1997) found SE to be highly 
correlated with student IM, while Schunk & Hanson found that increases in student effort and rate of performance 
increased with higher SE (1987). 

Extrinsic motivation (EM) also has merit in college level coursework as students learn for recognition, 
grades, and other performance metrics. However, the EM student has many drawbacks such as minimal effort to 
achieve goals, stopping of learning processes once goal is achieved, lower SE, and less cooperative behavior (Lei). 
Bandura (1977) hypothesized that self-efficacy affects an individual's choice of activities, level of effort, and 
persistence in completing tasks. Bandura states that people who have a low sense of efficacy for accomplishing a 
task may avoid it all together. On the other hand, individuals with high SE are noticed to work harder and endure 
longer when they meet complications (Ibid). For these reasons, our research is aimed at increasing SE and IM 
though the usage of gamificiation. 

Gamification 

Gamification is ELT pedagogy. In the most simplistic terms, gamification in education is the use of game 
components such as score, challenge, and achievement to learning objectives, in an effort to motivate and engage the 
student (Deterding et al., 2011). Methods deployed range from capture the flag events, quiz challenges, debate, use 
of leader boards and point gathering. The options for gamification are limited only by instructor creativity and 
trigger more efficient and engaged learning behavior (Muntean, 2011). 
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Competition, either against one’s self or against a peer group is the driving force behind the success of 
gamification. The learning effects of competition are based in social interaction and student desire to see their 
name at the top of a scored list (Conklin, 2006), or by comparing their personal score to peer achievement. Extrinsic 
motivation rewards, achieved through deliberate competitive goals, are an effective tool until intrinsic motivation 
can evolve (Rimm, 1986). As students begin to grow in self-efficacy the scoreboard will provide the metric that 
makes competition an effective learning vehicle. Students will be exposed to learning pace, comparative analysis, 
and experience motivation to move deeper into a topic. According to Lam, Yim and Chueng, competition has a 
constructive effect on participation and knowledge forming resulting in higher learning through social pressure to 
achieve (2004). 

Research questions 

• Does gamification pedagogy in IA coursework increase intrinsic motivation compared to didactic 
pedagogy? 

• Does gamification pedagogy in IA coursework increase self-efficacy compared to didactic pedagogy? 

RESEARCH MODEL 
Method 


The focus of the study was to understand the perspective of the students involved in the study. Thus, a 
direct observation method using loose interviewing was chosen as a qualitative method. The researcher was 
embedded into class exercises and generally performed the same function as others in the setting, so as to gain a 
deeper understanding of the student perceptions through comments and observation (Given, 2008). The participant 
observation method is typically accomplished by using a variety of methods including direct observation, informal 
interviewing, and open discussion (Given, 2008; Jorgensen, 1989). The researcher was embedded in other class 
assignments to build trust and help mitigate the issue of changing student behavior caused by researcher presence. 
Interviewing was performed in the format of discussion, or group conversation, to avoid a structure that can lead to 
bias (Krueger, 1998; Rubin et al. 2004). The group responses were then analyzed to identify key themes. A word 
tree was developed in order to visualize the most commonly used words and to examine the connections between 
IM, EM, and SE. Themes were identified and compared to already accepted tools that measure IM, EM and SE. 
The gamification classes were taught using the outlined “games” below, while the didactic class had lectures only. 
We discussed competition and score boarding after both courses completed their work. 

Participation 

Participants included students from two sections of an introduction to computer networking course (210) 
and two sections of a systems administration (344) course in a large public university in the Northern United States. 
A total of 96 students participated in the research. 

Class set up: IA210 

The 210 classes were first taught didactic methods of solving complex subnet problems. This involved 
algebraic computation and problem solving to write out possible network addresses. The didactic exercise ended 
here. Students in the gamification section continued the lesson of subnetting and were taught how to use Nmap, a 
tool used for scanning subnets. Students used the gathered Nmap information to find clues hidden on the network. 

Gamification setup 

Students were provided an IP address of a node on a given network and expected to calculate the subnet 
mask, range of available IP addresses and then use Nmap to find the running HTML service. 
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For example, given the IP address 10.1.5.50/29, students had to use subnet math (no on-line calculators) to 
decipher the following: 

Subnet Mask: 255.255.255.240 
Subnet number: 6 

Available IP range: 10.1.5.49 -10.1.5.54/29 

Students then had to change the static IP of their own machine to the appropriate network, then Nmap the 
appropriate range to find the web server. Once found, the web server address could be typed into a browser where 
the next node address would be revealed along with a clue about a selected famous individual. 

Seven web servers were configured using VMware ESXI server, each in a different subnet and each with a 
different set of clues. Clues included the IP address of a node on the next network. As students progressed through 
the seven web servers, they were instructed to switch between Windows 7 and CentOS clients. 

The whiteboard in the front of the room showed a scoreboard of how many students had discovered the 
clues and a tally of how many peers had found all seven web servers. The hunt for the clues and the knowledge of 
peer success motivated the classroom to endeavor to finish the task. Competition has a constructive effect on 
participation and learning that will result in higher learning through social pressure to achieve (Burguillo, 2010). 

Objectives of the subnet lesson for both didactic and gamification: 

• Subnet a network 

• Solve given nodes for subnet data (subnet number, mask, range, broadcast) 

• Change static IP addresses in multiple operating systems 

• Use NMap to identify running services on a network 

Class set up: IA 344 

IA 344 is not only a Microsoft Server administration class, but also a security class. Good security comes 
down to offense and defense, hackers and protectors, ultimately culminating in managing the risk presented by 
security issues (Wei, 2010). Students are taught via Socratic/didactic pedagogy; that is the lectures on security for 
the operating system are done via debate and discussion. 

Students build Active Directory servers in VMware and install live network services such as DNS, NAP, 
NAC, DHCP and GPO. For this ELT lab, students are asked to build a working DNS and DHCP server that 
distributes IP addresses in a given scope. The work is then graded for administrative quality and correctness. This 
is done for both didactic and gamification deliveries. 

For the gamification lesson and after completion of the administrative task, students are then asked to read 
a case study of a Microsoft breached server event. The case study does not reveal the root cause of the breach. 
Students are then asked to restore VMware snapshot 1 which is a replica of the server breached in the case study. 
Next, the student is tasked to review the new server and analyze the server security to solve the case study. They 
are told that there a minimum of five areas that will lead them to root cause. 

These are the objectives of the gamification assignment (not shared with student): 

• Attack surface - Web and FTP are installed, but not needed 

• Firewall rules - should remove unused and constrain others 

• Remove unused protocols/ports 

• Review audit logs 

• Verify User least privilege 
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As students find security issues, they are asked to write them down for future grading and fix the issue. A 
public scoreboard tracks how many holes have been identified and by how many students. Upon successful 
completion of the security review, students are asked to write a brief root cause analysis and deliver to the 
instructor. Snapshot 2 can then be restored to allow the student to continue to work on a new problem, while other 
peers complete snapshot 1. When all are finished, class discussion identifies the root cause and prevention methods. 
Each of the security holes is discussed, along with any other errors found by students. 

Interview/coded theme Results 

The student comments were collected during the exercise. Careful notes were taken for later coding of 
themes. Other data was collected at the completion of class in open discussion and coded for the similar themes. A 
word tree was built from the student comments to identify key themes. Both the gamification and didactic classes 
were asked the same open questions during the discussion: 

• What is your feeling of the assignment? 

• Do you feel that you have a firm grasp on the knowledge objectives? 

• Do you feel you could “figure out” new problems in Administration? 

• Which part of the lesson did you find most interesting? 

• Would/did the scoreboard play a role in your work? 

• Would/Did competition play a role in your work? 

The questions above were designed to be conversation starters; careful attention was paid to insure no the 
conversation was not led in a prejudiced direction. Two faculty members participated in the discussion, one 
managing the dialogue while the other monitored for bias. 

The data was then reviewed looking for themes and collating responses. For example, for the question 
“what is your feeling on the assignment...” students that answered cool, fun, or loved it, were all coded as positive 
response under the category of experiencing pleasure from the task. Lei’s model of measuring Intrinsic and 
extrinsic motivation was used as the accepted measuring tool (2010). 

DISCUSSION 

In this study, we sought to test the usefulness of gamification as a teaching pedagogy in Information 
Assurance classes to increase intrinsic motivation and self-efficacy in students. This was a qualitative study on the 
usage of gamification as the underpinning of experiential learning theory (ELT) as the tool used to increase intrinsic 
motivation (IM) and self-efficacy (SE) to perform IA tasks. 

Intrinsic/Extrinsic motivation 

As seen in the student responses, intrinsic motivation increased dramatically with the introduction of 
gamification. The course taught with didactic pedagogy had only 2 students who found the exercise fun and only 3 
students who were able to organize knowledge and relate it to existing knowledge (Lei, 2010). In the exercises that 
were taught using gamification, 25 found the work fun and 56 were able to tie the work to previous knowledge. An 
alarming rate of 92.2% of the students exposed to gamification responded in intrinsic motivation themes. Only 
30.5% of the didactic students where intrinsically motivated. Burguillo states clearly that intrinsic motivation is a 
key element to achieved learning. Increased IM will prompt the student to engage in exploration, effort, and 
participation driven by curiosity and not explicit reward (2010). 

Extrinsic motivation, whereby the student is motivated by grade, least effort needed, and external sources 
of motivation revealed themes that were an expectant opposite. A miniscule .078% of the gamification students 
were motivated extrinsically, while the didactic class was .70% externally motivated. 
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Table 1: Intrinsic Motivation 


Didactic 

N=42 

Gamification 

N=54 

“Word tree” examples; Coded Comments 

Intrinsic Themes Identified bv Lei (2010) 

2 

25 

“Fun” 

Experience pleasure in what they are doing 

21 

32 

Questions why a setting was made 

Attends to instruction 

2 

43 

“Tough, but cool” 

Perseverance 

0 

45 

“Another issue down” 

Applies skills and knowledge to problem 

0 

25 

Tried “X” 

Show creativity in action 

12 

44 

Why? What if? 

Striver for true understanding 

0 

35 

“High score!!” 

Regular evaluation or monitoring of own 
progress 

3 

11 

“This wasn’t covered in lecture” 

Organize knowledge and relate it to existing 
knowledge 

23 

74 

“How have people found 5 errors” 

Regular evaluation or monitoring of own 
progress 

3 

56 

“Now the lecture makes sense” 

Organize knowledge and relate it to existing 
knowledge 

3 

66 

“I found more then 5 errors” 

Undertake more challenging aspects of a task 

2 

43 

“Can we do more of this?” 

Does not depend on tangible reward 


Table 2: Extrinsic Motivation 


Didactic 

N=42 

Gamification 

N=54 

“Word tree” examples; Coded Comments 

Extrinsic Themes Identified bv Lei (2010) 

27 

4 

" I could not evaluate and find errors in a 
running system" 

Low self-esteem 

16 

7 

"I'd try it for Extra credit" 

Receiving extremely rewards or 
reinforcements (e.g. extra credit or bonus 
points) 

22 

5 

"Competition WOULD have been important 
piece" 

Competition for tangible rewards (e.g. honors 
and awards) 

31 

4 

"I'd do the work as a class" 

Social reasons for learning 

12 

5 

Stupid/dumb, when will I use this 

Learning compliance 

31 

14 

"Scoreboard WOULD have helped me" 

Competition for tangible rewards (e.g. honors 
and awards) 

23 

3 

"Would we have to find all 5 errors for 
credit?" 

Least effort needed 


Self-efficacy 

To measure self-efficacy we used the seminal chapter from Zimmerman’s book Adolescents' development 
of personal agency: The role of self-efficacy beliefs and self-regulatory skill (2006). More specifically, we used 
Albert Bandura’s chapter Guide for Constructing Self-efficacy Scales, which has been cited in other research over 
1300 times (1997). Efficacy items should accurately reflect the construct. Self-efficacy is concerned with perceived 
capability. According to Bandura, the items should be phrased in terms of can do rather than will do as can is a 
judgment of capability; will is a statement of intention (1997). 

Discussion questions/statements to study self-efficacy included: 

• Can you figure out any assignment in Windows server? 

• Can you finish assignments for 344 completely and on time? 

• I feel I can accomplish untaught administration tasks. 

• I feel I can help others with the course. 

Students were asked to rank the comfort levels in each of the themes above. Specifically, the students were 
asked to pick a quartile 0-100, and then provide their personal rank within the quartile. 
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Self-efficacy (SE) is as personal confidence that one has to execute strategy to attain designated goals 
(Bandurra). Zimmerman (1997) found SE to be highly correlated with student intrinsic motivation, while Schunk & 
Hanson found that increases in student effort and rate of performance increased with higher SE (1987). In our study, 
we found that student SE soared while being taught using gamification pedagogy. Consider the student self- 
confidence in the statement, “I can figure out how to do anything in Windows server”. In the gamification course, 
90.3% of students answered in the 50-100 range, while 28.5% of the didactic class answered in the same range on 
the same statement. SE is a key ingredient to creating aptitude and gamification is a pedagogy that dramatically 
increases SE. 


Table 3: Self-efficacy 


Gamification N=31 


Didactic N=28 

Responses 

Mean Score 

Quartile 

Bandura themes (1977) 

Responses 

Mean Score 

0 

0.0 

0-25 

"I can figure out how to 
do anything in Windows 
server" 

7 

21.2 

3 

37.3 

26-50 

13 

38.5 

10 

17.1 

51-75 

6 

62.2 

18 

92.4 

76-100 

2 

80.2 


Gamification N=31 


Didactic N=28 

Responses 

Mean Score 

Quartile 

Bandura themes (1977) 

Responses 

Mean Score 

0 

0.0 

0-25 

’ I can finish assignments 
in 344 on time by 
deadline" 

0 

0.0 

0 

0.0 

26-50 

7 

41.3 

16 

61.4 

51-75 

18 

54.0 

15 

93.2 

76-100 

6 

81.2 


Gamification N=31 


Didactic N=28 

Responses 

Mean Score 

Quartile 

Bandura themes (1977) 

Responses 

Mean Score 

1 

20.0 

0-25 

" I feel I have the 
aptitude to evaluate 
"untaught" 
administration tasks 

4 

24.0 

3 

39.9 

26-50 

18 

48.4 

15 

65.5 

51-75 

4 

35.5 

12 

91.1 

76-100 

2 

81.6 


Gamification N=31 


Didactic N=28 

Responses 

Mean Score 

Quartile 

Bandura themes (1977) 

Responses 

Mean Score 

2 

22.2 

0-25 

" I can help others with 
course tasks" 

3 

4.2 

7 

44.2 

26-50 

6 

48.8 

17 

67.8 

51-75 

14 

51.5 

5 

93.1 

76-100 

5 

78.8 
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